Sampling-based optimal kinodynamic planning with motion primitives

This paper proposes a novel sampling-based motion planner, which integrates in RRT* (Rapidly exploring Random Tree star) a database of pre-computed motion primitives to alleviate its computational load and allow for motion planning in a dynamic or partially known environment. The database is built by considering a set of initial and final state pairs in some grid space, and determining for each pair an optimal trajectory that is compatible with the system dynamics and constraints, while minimizing a cost. Nodes are progressively added to the tree {of feasible trajectories in the RRT* by extracting at random a sample in the gridded state space and selecting the best obstacle-free motion primitive in the database that joins it to an existing node. The tree is rewired if some nodes can be reached from the new sampled state through an obstacle-free motion primitive with lower cost. The computationally more intensive part of motion planning is thus moved to the preliminary offline phase of the database construction at the price of some performance degradation due to gridding. Grid resolution can be tuned so as to compromise between (sub)optimality and size of the database. The planner is shown to be asymptotically optimal as the grid resolution goes to zero and the number of sampled states grows to infinity

Following Newton direction in Policy Gradient with parameter exploration

This paper investigates the use of second-order methods to solve Markov Decision Processes (MDPs). Despite the popularity of second-order methods in optimization literature, so far little attention has been paid to the extension of such techniques to face sequential decision problems. Here we provide a model-free Reinforcement Learning method that estimates the Newton direction by sampling directly in the parameter space. In order to compute the Newton direction we provide the formulation of the Hessian of the expected return, a technique for variance reduction in the sample-based estimation and a finite sample analysis in the case of Normal distribution. Beside discussing the theoretical properties, we empirically evaluate the method on an instructional linear-quadratic regulator and on a complex dynamical quadrotor system

Using Modelica for advanced Multi-Body modelling in 3D graphical robotic simulators

This paper describes a framework to extend the 3D robotic simulation environment Gazebo, and similar ones, with enhanced, tailor-made, multi-body dynamics specified in the Modelica language. The body-to-body interaction models are written in Modelica, but they use the sophisticated collision detection capabilities of the Gazebo engine. This contribution is a first step toward the simulation of complex robotics systems integrating detailed physics modelling and realistic sensors such as lidar and cameras. A proof-of-concept implementation is described in the paper integrating Gazebo collider and the Modelica MultiBody library, and the results obtained when simulating the interaction of an elastic sphere with a rigid plane are shown

Towards the Implementation of a MPC-based Planner on an Autonomous All-Terrain Vehicle

Planning and control for a wheeled mobile robot are challenging problems when poorly traversable terrains, including dynamic obstacles, are considered. To accomplish a mission, the control system should firstly guarantee the vehicle integrity, for example with respect to possible roll-over/tip-over phenomena. A fundamental contribution to achieve this goal, however, comes from the planner as well. In fact, computing a path that takes into account the terrain traversability, the kinematic and dynamic vehicle constraints, and the presence of dynamic obstacles, is a first and crucial step towards ensuring the vehicle integrity. The present paper addresses some of the aforementioned issues, describing the hardware/software architecture of the planning and control system of an autonomous All-Terrain Mobile Robot and the implementation of a real-time path planner

Single injection dual phase CBCT technique ameliorates results of trans-arterial chemoembolization for hepatocellular cancer

Cone-beam CT (CBCT) application to the field of trans-arterial chemoembolization has been recently the focus of several researches. This imaging modality is performed with a rotation of the C-arm around the patient, without needs of patient repositioning. Datasets are immediately processed, obtaining volumetric CT-like images with the possibility of post-processing and reconstruction of images. Dual phase CBCT recently introduced in clinical practice consists in a first arterial acquisition followed by a delayed acquisition corresponding to a venous phase. The introduction of this feature has overcome the limit of single-phase acquisitions, allowing lesions characterization. Moreover these recent advantages have several intra-procedural implications. Detailed technical and acquisition parameters will be widely exposed in this review with particular attention to: catheter positioning, acquisition delay, injection parameters, patient positioning and contrast dilution. Comparison with standard of practice second line imaging [multidetector computer tomography (MDCT) and MDCT/arteriography] demonstrate the capability of detecting occult nodules providing some clinical implications thus potentially identifying a sub set of patients with aggressive disease behaviour. Other intra-procedural advantages of dual phase CBCT usage consist in a better tumor feeder visualization, reduction of proper DSA and fluoroscopic time, suggestion the presence of an extrahepatic parasitic feeder thus resulting in a more accurate treatment. Finally, the volumetrical intraprocedural evaluation of accumulation of embolic agent has proved to be correlate with treatment response if compared with MRI

Poli-RRT*: optimal RRT-based planning for constrained and feedback linearisable vehicle dynamics

This paper proposes a Rapidly exploring Random Trees planning strategy (Poli-RRT*) that computes optimal trajectories in presence of vehicle constraints (e.g., differential and actuation constraints) without approximating the nonlinear dynamics, but relying on exact linearisation. In this way, the optimal control problem that is introduced to determine the trajectories extending the tree can be expressed as a quadratic program and efficiently solved. Poli-RRT* is formulated and tested via simulation on a unicycle-like model of a vehicle subject to actuation constraints. Notably, the approach can be applied to any other feedback linearisable vehicle model, subject to different types of constraints

Policy gradient in Lipschitz Markov Decision Processes

This paper is about the exploitation of Lipschitz continuity properties for Markov Decision Processes to safely speed up policy-gradient algorithms. Starting from assumptions about the Lipschitz continuity of the state-transition model, the reward function, and the policies considered in the learning process, we show that both the expected return of a policy and its gradient are Lipschitz continuous w.r.t. policy parameters. By leveraging such properties, we define policy-parameter updates that guarantee a performance improvement at each iteration. The proposed methods are empirically evaluated and compared to other related approaches using different configurations of three popular control scenarios: the linear quadratic regulator, the mass-spring-damper system and the ship-steering control

Flat-RRT*: A sampling-based optimal trajectory planner for differentially flat vehicles with constrained dynamics

This paper introduces the \algo\space algorithm, which is a variant of the optimal Rapidly exploring Random Tree RRT* planner, accounting for actuation constraints on the vehicle dynamics in the optimal trajectory design. The proposed algorithm is applicable to vehicles that can be modelled with differentially flat dynamics, like unicycle and bicycle kinematics. The main idea is to exploit the flatness property so as to finitely parametrize trajectories, and design a set of motion primitives that represent optimal constrained trajectories joining two configurations in a grid space. A procedure to determine constrained (though sub-optimal) trajectories joining arbitrary configurations based on the motion primitives is then proposed. This eases and accelerates the construction of the tree to the purpose of online trajectory (re)planning in an uncertain environment, where the obstacle map may be continuously updated as the vehicle moves around, or unexpected events may occur and alter the free configuration space

Estimating a mean-path from a set of 2-d curves

To perform many common industrial robotic tasks, e.g. deburring a work-piece, in small and medium size companies where a model of the work-piece may not be available, building a geometrical model of how to perform the task from a data set of human demonstrations is highly demanded. In many cases, however, the human demonstrations may be sub-optimal and noisy solutions to the problem of performing a task. For example, an expert may not completely remove the burrs that result in deburring residuals on the work-piece. Hence, we present an iterative algorithm to estimate a noise-free geometrical model of a work-piece from a given dataset of profiles with deburring residuals. In a case study, we compare the profiles obtained with the proposed method, nonlinear principal component analysis and Gaussian mixture model/Gaussian mixture regression. The comparison illustrates the effectiveness of the proposed method, in terms of accuracy, to compute a noise-free profile model of a task